In this paper, we target at the problem of learning a generalizable dynamic radiance field from monocular videos. Different from most existing NeRF methods that are based on multiple views, monocular videos only contain one view at each timestamp, thereby suffering from ambiguity along the view direction in estimating point features and scene flows. Previous studies such as DynNeRF disambiguate point features by positional encoding, which is not transferable and severely limits the generalization ability. As a result, these methods have to train one independent model for each scene and suffer from heavy computational costs when applying to increasing monocular videos in real-world applications. To address this, We propose MonoNeRF to simultaneously learn point features and scene flows with point trajectory and feature correspondence constraints across frames. More specifically, we learn an implicit velocity field to estimate point trajectory from temporal features with Neural ODE, which is followed by a flow-based feature aggregation module to obtain spatial features along the point trajectory. We jointly optimize temporal and spatial features by training the network in an end-to-end manner. Experiments show that our MonoNeRF is able to learn from multiple scenes and support new applications such as scene editing, unseen frame synthesis, and fast novel scene adaptation.

With the rising industrial attention to 3D virtual modeling technology, generating novel 3D content based on specified conditions (e.g. text) has become a hot issue. In this paper, we propose a new generative 3D modeling framework called Diffusion-SDF for the challenging task of text-to-shape synthesis. Previous approaches lack flexibility in both 3D data representation and shape generation, thereby failing to generate highly diversified 3D shapes conforming to the given text descriptions. To address this, we propose a SDF autoencoder together with the Voxelized Diffusion model to learn and generate representations for voxelized signed distance fields (SDFs) of 3D shapes. Specifically, we design a novel UinU-Net architecture that implants a local-focused inner network inside the standard U-Net architecture, which enables better reconstruction of patch-independent SDF representations. We extend our approach to further text-to-shape tasks including text-conditioned shape completion and manipulation. Experimental results show that Diffusion-SDF is capable of generating both high-quality and highly diversified 3D shapes that conform well to the given text descriptions. Diffusion-SDF has demonstrated its superiority compared to previous state-of-the-art text-to-shape approaches.

有效地保留和编码结构功能从不规则和稀疏点点中的对象中的对象是对点云上3D对象检测的关键挑战。最近，变形金刚在许多2D甚至3D视觉任务上都表现出了有希望的表现。与固定和刚性卷积内核相比，变压器中的自发机制可以适应地排除无关或嘈杂点，因此适合保留不规则的LIDAR点云中的局部空间结构。但是，Transformer仅根据自我发项机制对点特征执行简单的总和，所有点具有相同的价值变换。这种各向同性操作缺乏捕获面向方向距离的局部结构的能力，这对于3D对象检测很重要。在这项工作中，我们提出了一个结构插入变压器（Seformer），它不仅可以将本地结构保存为传统变压器，而且还可以编码本地结构。与传统变压器中的自我发挥机制相比，Seformer基于与查询点的相对方向和距离学习了价值点的不同特征变换。然后，我们提出了一个基于Seformer的网络，用于高性能3D对象检测。广泛的实验表明，所提出的体系结构可以在Waymo Open Datatet上实现SOTA结果，这是自动驾驶的最大3D检测基准。具体而言，Seformer获得79.02％的地图，比现有作品高1.2％。我们将发布代码。

Talking Head Synthesis是一项新兴技术，在电影配音，虚拟化身和在线教育中具有广泛的应用。最近基于NERF的方法会产生更自然的会话视频，因为它们更好地捕获了面部的3D结构信息。但是，需要使用大型数据集对每个身份进行特定模型。在本文中，我们提出了动态面部辐射场（DFRF），以进行几次交谈的头部综合，这可以在很少的训练数据中迅速概括为看不见的身份。与现有的基于NERF的方法不同，该方法将特定人的3D几何形状和外观直接编码到网络中，我们的DFRF条件面对2D外观图像上的辐射场，以便先验学习面部。因此，可以通过很少的参考图像灵活地调整面部辐射场。此外，为了更好地对面部变形进行建模，我们提出了一个在音频信号条件下的可区分面翘曲模块，以使所有参考图像变形到查询空间。广泛的实验表明，只有数十秒钟的训练剪辑可用，我们提出的DFRF可以合成天然和高质量的音频驱动的会说话的头视频，用于只有40k迭代的新身份。我们强烈建议读者查看我们的补充视频以进行直观的比较。代码可在https://sstzal.github.io/dfrf/中找到。

我们提出了一种新颖的方法，可以可靠地估计相机的姿势，并在极端环境中获得的一系列图像，例如深海或外星地形。在这些挑战性条件下获得的数据被无纹理表面，图像退化以及重复性和高度模棱两可的结构所破坏。当天真地部署时，最先进的方法可能会在我们的经验分析确认的那些情况下失败。在本文中，我们试图在这些极端情况下使摄像机重新定位起作用。为此，我们提出：（i）一个分层定位系统，我们利用时间信息和（ii）一种新颖的环境感知图像增强方法来提高鲁棒性和准确性。我们广泛的实验结果表明，在两个极端环境下我们的方法有利于我们的方法：将自动的水下车辆定位，并将行星漫游者定位在火星样的沙漠中。此外，我们的方法仅使用20％的培训数据就可以在室内基准（7片数据集）上使用最先进的方法（7片数据集）实现可比性的性能。

在本文中，我们研究了深神经网络中的动态感知对抗攻击问题。大多数现有的对抗性攻击算法是在基本假设下设计的 - 网络架构在整个攻击过程中都是固定的。然而，这种假设不适用于许多最近提出的网络，例如最近提出的网络。 3D稀疏卷积网络，其中包含输入相关的执行，以提高计算效率。它导致严重问题的滞后梯度，由于架构之后的架构而导致当前步骤的学习攻击无效。为了解决这个问题，我们提出了一种带有铅梯度法（LGM）并显示出滞后梯度的显着影响。更具体地说，我们重新制定了梯度，以了解网络架构的潜在动态变化，使得学习攻击更好地“引导”的下一步，而是当网络架构动态变化时的动态 - 不知道方法。关于各种数据集的广泛实验表明，我们的LGM在语义细分和分类上实现了令人印象深刻的性能。与动态无知的方法相比，LGM在SCANNET和S3DIS数据集上均达到约20％的MIOU。 LGM还优于最近的点云攻击。

我们提出了一个框架，以便不断学习以对客观的视觉学习和理解为中心的表示。现有的对象形式依赖于个性化场景中的对象的监督，或者执行无监督的解剖学，这几乎无法处理现实世界中的复杂场景。为了减轻注释负担并放宽对数据统计复杂性的限制，我们的方法利用相互作用，从而有效地在学习以特征对象的表示的同时有效地采样对象和相应的训练信号的不同变化。在整个学习过程中，对象以随机顺序逐一流动，具有未知的身份，并且与可以通过卷积高度合成每个对象的潜在权重的潜在代码相关联。此外，采用了学习对象的重新识别和遗忘预防，以使学习过程有效且坚固。我们对拟议框架的关键特征进行了广泛的研究，并分析了学习的表示的特征。此外，我们展示了所提出的框架在学习表示中可以提高下游任务中的标签效率的能力。我们的代码和培训的型号将公开可用。

大多数现有的点云实例和语义分割方法在很大程度上依赖于强大的监督信号，这需要场景中每个点的点级标签。但是，这种强大的监督遭受了巨大的注释成本，引起了研究有效注释的需求。在本文中，我们发现实例的位置对实例和语义3D场景细分都很重要。通过充分利用位置，我们设计了一种弱监督的点云分割算法，该算法仅需要单击每个实例以指示其注释的位置。通过进行预处理过度分割，我们将这些位置注释扩展到seg级标签中。我们通过将未标记的片段分组分组到相关的附近标签段中，进一步设计一个段分组网络（SEGGROUP），以在SEG级标签下生成点级伪标签，以便现有的点级监督的分段模型可以直接消耗这些PSEUDO标签为了训练。实验结果表明，我们的SEG级监督方法（SEGGROUP）通过完全注释的点级监督方法获得了可比的结果。此外，在固定注释预算的情况下，它的表现优于最近弱监督的方法。

Non-line-of-sight (NLOS) imaging aims to reconstruct the three-dimensional hidden scenes from the data measured in the line-of-sight, which uses photon time-of-flight information encoded in light after multiple diffuse reflections. The under-sampled scanning data can facilitate fast imaging. However, the resulting reconstruction problem becomes a serious ill-posed inverse problem, the solution of which is of high possibility to be degraded due to noises and distortions. In this paper, we propose two novel NLOS reconstruction models based on curvature regularization, i.e., the object-domain curvature regularization model and the dual (i.e., signal and object)-domain curvature regularization model. Fast numerical optimization algorithms are developed relying on the alternating direction method of multipliers (ADMM) with the backtracking stepsize rule, which are further accelerated by GPU implementation. We evaluate the proposed algorithms on both synthetic and real datasets, which achieve state-of-the-art performance, especially in the compressed sensing setting. All our codes and data are available at https://github.com/Duanlab123/CurvNLOS.

In this paper, we propose a large-scale language pre-training for text GENeration using dIffusion modEl, which is named GENIE. GENIE is a pre-training sequence-to-sequence text generation model which combines Transformer and diffusion. The diffusion model accepts the latent information from the encoder, which is used to guide the denoising of the current time step. After multiple such denoise iterations, the diffusion model can restore the Gaussian noise to the diverse output text which is controlled by the input text. Moreover, such architecture design also allows us to adopt large scale pre-training on the GENIE. We propose a novel pre-training method named continuous paragraph denoise based on the characteristics of the diffusion model. Extensive experiments on the XSum, CNN/DailyMail, and Gigaword benchmarks shows that GENIE can achieves comparable performance with various strong baselines, especially after pre-training, the generation quality of GENIE is greatly improved. We have also conduct a lot of experiments on the generation diversity and parameter impact of GENIE. The code for GENIE will be made publicly available.